Adding support for .gitignore file like ignorefiles (and fix Windows build) by WolverinDEV · Pull Request #302 · sourcefrog/conserve

WolverinDEV · 2026-05-20T21:50:50Z

Hey ho,
it’s me again.

As the title suggests, this PR adds support for .gitignore-style ignore files.

Additionally combines two minor fixes / features:

It restores Windows build support, as the previous async-related changes accidentally broke the mount feature there.
It skips bucket region detection if it has been specified via an environment variable. This allows for custom self hosted S3 implementations like RustFS.

…er struct

sourcefrog

That's a great feature idea, and thanks for fixing the Windows build. Microsoft don't seem to be making it easy for me to have a working dev vm, but I don't mind taking patches for it.

sourcefrog · 2026-05-21T01:40:17Z

 }

+impl PathFilterOptions {
+    pub fn create_path_filter(&self) -> Result<Exclude> {


should we call this to_exclude?

Personally I would prefer to rename Exclude to PathFilter as that name seems more appropriate for what it actually does now. However, that is something I wanted to discuss before proposing the change, since keeping the naming consistent would require renaming quite a few places throughout the codebase.

Yes that sounds like an improvement, but let's consider it separately

WolverinDEV · 2026-05-21T07:44:05Z

Microsoft don't seem to be making it easy for me to have a working dev vm, but I don't mind taking patches for it.

Yeah, I saw the README. That's understandable. Sadly, I can't really provide any resources here either, as I'm personally on the very limited end as well 😆

In regards to your two comments, there is a significant distinction between .gitignore files and this ignore file implementation. This implementation does only implement the pattern format of .gitignore files. Actual (Git) ignore files within the source tree are intentionally ignored.

The motivation behind this is the following:
Providing an ignore file that supports general exclusion rules together with explicit exceptions allows for a clear specification of which files should be backed up (see the example below). Another possible approach would have been some kind of "include only" syntax, but that seemed both more limited and harder to implement cleanly.

I do not, however, want conserve as a backup tool to unintuitively respect in-tree ignore files. Personally, even skipping directories containing CACHEDIR.TAG by default is already somewhat at the edge. In my opinion, a backup tool should, by default, back up the actual filesystem as it exists unless explicitly configured otherwise.

Example ignorefile config

# This is an excerpt of the ignore file I use for my server.
# Due to limited backup storage, I want fine-grained control over which data is actually backed up
# (e.g. databases and configuration files, but not cache or log files).

# Ignore everything but make the /home rules overwriteable
/*
!/home
/home/*

# Vaultwarden data
!/home/vaultwarden
/home/vaultwarden/vw-data/icon_cache
/home/vaultwarden/vw-data/tmp

# Traefik
!/home/traefik
/home/traefik/plugins-local

# Vikunja
!/home/vikunja
</details>

sourcefrog · 2026-05-23T14:38:23Z

Microsoft don't seem to be making it easy for me to have a working dev vm, but I don't mind taking patches for it.

Yeah, I saw the README. That's understandable. Sadly, I can't really provide any resources here either, as I'm personally on the very limited end as well 😆

In regards to your two comments, there is a significant distinction between .gitignore files and this ignore file implementation. This implementation does only implement the pattern format of .gitignore files. Actual (Git) ignore files within the source tree are intentionally ignored.

Oh, OK, I see. Let's clarify that in the help text.

The motivation behind this is the following: Providing an ignore file that supports general exclusion rules together with explicit exceptions allows for a clear specification of which files should be backed up (see the example below). Another possible approach would have been some kind of "include only" syntax, but that seemed both more limited and harder to implement cleanly.

That makes sense.

I do not, however, want conserve as a backup tool to unintuitively respect in-tree ignore files. Personally, even skipping directories containing CACHEDIR.TAG by default is already somewhat at the edge. In my opinion, a backup tool should, by default, back up the actual filesystem as it exists unless explicitly configured otherwise.

I agree, even though build products can be large, there's a real risk of accidentally excluding something that you wish you'd kept. Possibly even the cachedir handling should be off by default.

sourcefrog · 2026-05-23T14:39:29Z

-        exclude: Vec<String>,
-        #[arg(long, short = 'E')]
-        exclude_from: Vec<String>,
-        #[arg(long = "only", short = 'i')]


I think you erroneously deleted this line which is causing the test failures.

🤦 how the hell did I miss that line in my own review of the changes....
Good catch!

Short option names must be unique for each argument, but '-i' is in use by both 'ignore_file' and 'only_subtree'

Well that had be a very funny but yet unlucky circumstance.
Not sure how we should handle this conflict in this case.

sourcefrog · 2026-05-24T15:32:47Z

+    exclude_from: Vec<String>,
+
+    /// Specify a gitignore like ignorefile
+    #[arg(long, short)]


Let's just specify short = 'G' or something here, so that it doesn't default to i? Or even just leave off short for now.

sourcefrog · 2026-05-24T15:35:22Z


+impl PathFilterOptions {
+    pub fn to_exclude(&self) -> Result<Exclude> {
+        if let Some(file) = &self.ignore_file {


I think we could potentially allow both and have them stack, but it doesn't have to be in this PR.

sourcefrog · 2026-05-24T15:37:17Z

+    }
+
+    pub fn from_ignorefile(file: impl AsRef<Path>) -> Result<Exclude> {
+        let mut builder = GitignoreBuilder::new("/");


I think the effect of this is that the root of the backup is matched by the root of gitignore patterns: in the same place that the git tree root would normally match? That makes sense.

sourcefrog · 2026-05-24T15:39:05Z

+    #[arg(long, short = 'E', conflicts_with = "ignore_file")]
+    exclude_from: Vec<String>,
+
+    /// Specify a gitignore like ignorefile


Suggested change

/// Specify a gitignore like ignorefile

/// Exclude paths that match rules from a file with gitignore syntax.

WolverinDEV added 5 commits May 20, 2026 23:46

fix: made conserve compile on Windows again

4a79ad3

refactor: abstracted away CLI exclude options into a shared path filt…

ffc8831

…er struct

feat: skip s3 bucket region detection if manually specified

a182b03

feat: adding support for git like ignore files

5dba308

misc: applied cargo fmt

e2b7cab

sourcefrog reviewed May 21, 2026

View reviewed changes

sourcefrog reviewed May 23, 2026

View reviewed changes

WolverinDEV added 2 commits May 24, 2026 11:08

fix: added mistakenly deleted line

78f8fcb

chore: renamed create_path_filter to to_exclude

541365c

WolverinDEV requested a review from sourcefrog May 24, 2026 09:09

sourcefrog requested changes May 24, 2026

View reviewed changes

	/// Specify a gitignore like ignorefile
	/// Exclude paths that match rules from a file with gitignore syntax.

Conversation

WolverinDEV commented May 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sourcefrog left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

WolverinDEV commented May 21, 2026

Uh oh!

sourcefrog commented May 23, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

WolverinDEV commented May 20, 2026 •

edited

Loading